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A Shared Vision of Excellence in the Classroom 


If Indiana schools are going to prepare students for success in the 21 st century, administrators 
and teachers need to work together toward a shared vision of excellence in the classroom. 
Indiana's innovative pilot evaluation systems give educators the tools and common language 
they need to have honest conversations about their most important job: helping students learn. 
This requires a sustained and meaningful culture shift at the school level, especially in the way 
we envision the relationship between teachers and administrators. Indiana's evaluation pilot is 
informed by three core principles: 


Teachers deserve to 
be treated like 
professionals. 


Current evaluations treat teachers like interchangeable parts — 
rating nearly all teachers good or great and failing to give 
teachers the accurate, useful feedback they need to do their best 
work in the classroom. Teachers deserve regular feedback on 
their performance, opportunities for professional growth, and 
recognition when they do exceptional work. 


Better evaluations 
give principals the 
tools they need to 
become instructional 
leaders. 


The new systems will help principals support their teachers by 
helping them accurately pinpoint teachers' strengths and 
weaknesses. Helping teachers reach their potential in the 
classroom is a principal's most important job as an instructional 
leader, and a new evaluation system will hold principals 
accountable for helping all their teachers learn and grow. 


When teachers grow, Novice and veteran teachers alike can look forward to detailed. 
Students grow. constructive feedback, tailored to the individual needs of their 

students. Teachers and principals will meet regularly to discuss 
successes and areas for improvement and set professional goals. 
The end result is better instruction for every student. 


Successful collaboration between administrators and teachers requires honesty, reflection, and a 
commitment to constant improvement. Educators of all experience levels will need to learn 
new skills and reevaluate how they spend their time. Meaningful change won't happen 
overnight, but throughout the evaluation pilot year we've seen administrators and teachers 
work together to make the kinds of instructional adjustments that can have a lasting, positive 
effect on student learning. 
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Looking Ahead to 2012-2013: Quality Feedback for All Indiana Teachers 


Nothing we can do for our students matters more than giving them effective teachers. Research 
has proven this time and again. We need to do everything we can to give all our teachers the 
support they need to do their best work, because when they succeed, our students succeed. 
Without effective evaluation systems, we can't identify and retain excellent teachers, provide 
useful feedback and support to help teachers improve, or intervene when teachers consistently 
perform poorly. 

With this in mind, in the spring of 2011 the Indiana legislature passed IC 20-28-11.5, a new law 
relating to the evaluation of all certified teaching staff and administrators. Prior to this 
legislation, evaluation systems around the state varied greatly in quality and consistency. The 
new law introduced three main requirements for all evaluation systems: 

• Every teacher must receive an evaluation annually; 

• Every evaluation system must include four performance categories: Highly Effective, 
Effective, Improvement Necessary, and Ineffective; and 

• Every evaluation system must incorporate measures of student growth and achievement 
as a significant portion of a teacher's evaluation. 

Since the mid-year pilot report was released, the DOE has been hard at work providing the 
information, support, and training that corporations need to successfully implement new 
evaluation systems in school year 2012-2013. The DOE 
website now features extensive walk-throughs of the 
legislative requirements, offering school corporations 
guidance on a number of topics such as developing an 
evaluation plan, incorporating rigorous measures of 
student learning, and making human resource decisions. 

Additionally, the DOE has offered monthly evaluator 
training through the Educational Service Centers (ESCs) 
for corporations adopting RISE next year, or for 
corporations that want to supplement local training. These 
trainings will continue throughout summer 2012 and 
beyond to assist corporations with implementation. 

Through a partnership with TNTP 1 , the DOE has continued its pilot of new evaluation systems 
in six corporations throughout the state. This report focuses on the results from the year-long 
pilot. The findings and recommendations are intended to assist corporations in the 
implementation of their new evaluation systems. Additionally, the DOE will use these and 
other pilot findings to improve RISE for the upcoming year. RISE version 2.0 will be released 
statewide in early August. 11 




The DOE has been 
hard at work, providing 
resources on 
implementation, design, 
and planning through the 
DOE guidance website, 
the mid-year pilot report, 
and the pages of the RISE 
website. 
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Indiana Evaluation Pilot - Update 


Six corporations piloted new evaluation systems in school year 2011-2012. Three of these 
corporations — Bloomfield School District, Fort Wayne Community Schools, and Greensburg 
Community Schools — piloted the RISE evaluation system. The other three corporations — 
Bremen Public Schools, Beech Grove City Schools, MSD Warren Township — piloted alternative 
evaluation systems. For a profile of each of these pilot corporations and further details on 
evaluation model design, please see Appendix A. 

DOE and TNTP released The Indiana Evaluation Pilot: Mid-Year Report and Recommendations in 
March 2012. Based on early findings from the pilot, that report outlines recommendations for 
corporations starting the evaluation system design and early-phase implementation. Reference 
the mid-year report for specific recommendations on forming an evaluation design committee, 
choosing components for an evaluation system, communicating to stakeholders, and providing 
effective training to evaluators. 


This current report will focus exclusively on implementation challenges and successes 
experienced during the pilot year. The report is designed to give helpful advice to corporations 
implementing new evaluation systems this upcoming school year. It is not intended to be an in- 
depth implementation or design guide. For additional information on implementation, design, 
and planning, visit the DOE guidance website, read the mid-year pilot report, and visit the 
Getting Started and Resources pages of the RISE website. 


Data Collection 

The findings and recommendations in this report are 
based on multiple sources of data collected throughout 
the pilot year. Teachers and evaluators in all six pilot 
corporations were surveyed three times - in August, 
January, and May - to understand their experiences 
with prior evaluation systems and the implementation 
of the new system in their respective corporations. 
Qualitative data was collected via individual interviews 
and focus groups with central office staff, evaluators, 
and teachers. 






Teachers and 
evaluators in all six 
pilot corporations were 
surveyed three times - 
in August, January, and 
May - to understand 
their experiences with 
prior evaluation 
systems and the 
implementation of the 
new system. 




Additional data was collected from RISE corporations 
to assess the success of the state evaluation system. Final professional practice ratings were 
collected and analyzed, as were Student Learning Objectives (SLOs). Scores for measures of 
student learning - SLOs, Individual Growth Model (IGM), and the A-F ratings needed to 
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determine School-Wide Growth (SWG) - were not yet available at the time of drafting this 
report. Summative evaluation scores cannot be collected or analyzed until these measures are 
scored. Therefore, the findings and recommendations in this report are based on the data that 
was available as of early June 2012. Guidance and recommendations for corporations around 
scoring and summative evaluation rating determination will be provided in Fall 2012. m Because 
more data was collected for RISE corporations, many of the findings and recommendations 
below are specific to RISE. 

To protect the confidentiality of pilot participants, data is presented in aggregate and does not 
identify specific schools, administrators, or teachers. 


Culture Shift: Redefining the Administrator/Teacher Relationship 


In successful schools, staff are focused on improving student outcomes above all else. Every 
day, in the classroom and in the hallways, actions are driven by the desire to give students the 
best education possible. As a result, all of the findings and recommendations in this report have 
one thing in common: they lead to a shift in culture at the school level that promotes improved 
student learning. Throughout this report, the term school culture refers to the environment of a 
school that results from the prevailing mindsets, beliefs, and actions of its administrators, 
teachers, and students and that contributes to, or detracts from, teaching excellence and student 
achievement. The term culture shift refers to the need for corporations and schools to create a 
collaborative atmosphere where school leaders and teachers work together toward a shared 

vision of instructional excellence, adjusting 
classroom practice and raising student achievement 
through observation, feedback, reflection, and 
improvement.^ 
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The term culture shift 
refers to the need for 
corporations and 
schools to create a 
collaborative 
atmosphere where 
school leaders and 
teachers work together 
toward a shared vision 
of instructional 
excellence... 


Many schools already embrace the type of culture 
needed for successful implementation, but for 
others, this will be a substantial shift in the way that 
educators view their roles and responsibilities. 
Successful implementation of a new evaluation 
system requires school leaders to view their role as 
the instructional leader of the school. That means 
spending the majority of their time in classrooms 
observing teaching and following-up with honest, 
actionable, and timely feedback. Similarly, teachers 
must be open to a level of feedback many have 
never received before, and be willing to adapt and improve their practice accordingly. This 
culture shift is essential in order for new evaluation systems to have the maximum impact on 
student learning. 
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Based on conclusions from the Indiana Teacher Effectiveness Pilot, this report aims to provide 
schools with advice on how to give teachers quality feedback on their job performance, which 
will refine their instruction throughout the year and improve student learning. The report is 
divided into two main sections. Findings and Recommendations, which outline the results of 
the pilot and provide advice to corporations based on those results, respectively. 

End-of-Year Findings 

1. Implementing a new evaluation system requires a substantial shift in professional 
expectations and responsibilities, but with practice the new approach will result in improved 
instruction and student learning. 

Indiana's teacher evaluation legislation requires all school staff to adjust to new professional 
expectations and responsibilities, and pilot results show that committed educators are up to 
these tasks. The improved rubrics outline clear expectations, and using those tools effectively 
requires more observations, higher-quality feedback, and more time spent setting and 
measuring student learning goals. Incorporating this new approach into the school day needs 
to be supported at the corporation level. Central office staff, particularly managers of building- 
level administrators, will need to support a consistent, corporation-wide culture shift by 
spending more time in schools. 

Overall, the pilot showed that there is a wide variety of 
ways that corporations and schools can successfully meet 
the requirements for observations, providing the quality 
feedback that will lead to better classroom instructions 
Teachers in the pilot report receiving, most commonly, 
four observations this year, except in one corporation 
where an expectation of short, frequent observations 
resulted in teachers most commonly receiving 35 or more 
observations this year. vi Teachers most commonly 
received two shorter observations (10-20 minutes) and 
two longer observations (40 minutes or more). Teachers 
in the corporation that required short, frequent 
observations report receiving most commonly 15 short observations (less than ten minutes), two 
10-20 minute observations, and two 20-30 minute observations. vii Most observations in all 
corporations were unannounced and followed by written feedback, and usually two of the 
observations were followed by post-conferences.™ Regardless of whether a corporation 
chooses to implement more frequent, shorter classroom visits or less frequent, longer visits, the 
core principle remains the same: increased observations and regular feedback improve 
classroom performance. 


/ ^ 

Teachers not only 

report receiving more 
frequent observations 
than in previous years, 
but also that they are 
getting timely, 
evidence-based 
feedback. 

% / 
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Pilot evaluators implemented the new observation and feedback protocols with fidelity 
throughout the past year. Teachers report receiving more frequent observations than in 
previous years, along with evidence-based feedback. Frequency of feedback nearly doubled, 
with teachers on average receiving written feedback after an observation 70 percent of the time, 
compared to only 37 percent of the time last year. ix Over two-thirds of teachers also report the 
feedback they received was always based on evidence or examples from observations (69%). x 

While important that the requirements of the new evaluation system were met, it is most 
encouraging that these evaluation activities have led to increased teacher and administrator 
accountability. The majority of teachers report taking a more active role in their evaluation 

(57%). xi This more active role has resulted in 
teachers feeling more accountable for student 
achievement than in the past (51%). xii These 
evaluation activities also heightened 
administrators' sense of accountability for the 
quality of instruction taking place in schools. More 
than 80% of administrators felt an increased sense 
of accountability for teacher and student 
performance under the new evaluation system, 
suggesting that increased time spent in the 
classroom helps focus administrators on their most 
important task: improving student learning 
schoolwide. 

Likewise, it is encouraging that the new evaluation 
system is also leading to an increase in teacher and administrator practices that promote better 
results for students. Teachers and administrators report using new, student-centered rubrics in 
a wide variety of ways to drive better, more consistent outcomes. About half of all teachers in 
the pilot corporations are utilizing the new rubrics for planning lessons on a day-to-day basis, 
reflecting on their instruction, and improving their practice. xiii Evaluators report they use the 
rubric to guide classroom observations (85%) xiv and to provide teachers feedback on strengths 
(96%) and specific areas for improvement aligned with the rubric (93%). xv 

Clearer, shared expectations for effective teaching are an important starting point for improving 
student outcomes, but teachers also need focused support in this challenging work. That means 
that administrators need to truly be instructional leaders. In the pilot, a majority of 
administrators report that the new systems have resulted in a significant shift in their 
responsibilities toward instructional leadership (58%), and eight in ten believe time spent on 
these activities is well-spent (85%). xvi More time is spent on observing teachers and providing 
feedback (89%) xvii , and less on discipline, compliance tasks, and building management/™ This 
is supported by a group of RISE pilot evaluators who report a significant shift toward 
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About half of all 
teachers in the pilot 
corporations are 
utilizing the new 
rubrics for planning 
lessons on a day-to-day 
basis, reflecting on 
their instruction, and 
improving their 
practice. 
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evaluation related activities (observation, mapping evidence to the rubric, conferencing, 
assessments) in how they spend their time. xix 


Chart 1: Administrators report spending 100% more time focused on teacher effectiveness. 



This group of evaluators report 21 percentage points more of their total time was spent on 
Evaluation and an additional 4 percentage points of their total time on professional 
development (as seen in Chart 1). Evaluators compensated for this increase mostly by 
decreasing the amount of time spent on administrative activities such as emails, paperwork, 
oversight, and operations. xx Evaluators, therefore, should anticipate spending a majority of 
their time in classrooms, observing instruction and ensuring that students are learning at a high 
level. 

2. Evaluators must develop observation and feedback skills to achieve the goals of the new 
system. 

Getting good results for students requires evaluators to be instructional leaders who provide 
evidence-based, actionable, and valuable information to teachers. To implement the new 
evaluation system successfully, evaluators must learn to differentiate teacher strengths and 
weaknesses and accurately evaluate teachers so they can continually develop them as 
professionals and improve student outcomes. 
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We know from talking to teachers and administrators that teachers in the pilot districts work 
hard and care about their students' results. Nevertheless, data show that there must be room 
for improvement and not all teachers are equally effective. Although at the time of writing we 
do not yet have individual growth model data, we know that only some teachers are able to 
achieve extraordinary gains with their students. xxi Disappointingly, the Teacher Effectiveness 
Rubric (TER) results from the pilots did not reflect this reality. Overall, 91% of teachers in the 
RISE pilots were given a rubric rating of Highly Effective or Effective. RISE evaluators rated 
teachers higher on end-of-year rubric ratings than they had anticipated. On an end-of-year 
survey, RISE evaluators indicated a teacher rating distribution of performance levels shown in 
Chart 2. Evaluators, however, assigned higher ratings in practice than expected in the survey; 
double the percentage expected in the Highly Effective category (15% compared to 30%) and 6 
percentage points more than expected in the Highly Effective and Effective categories 
combined. xxii 


Chart 2 Evaluators assigned much higher ratings in practice than they had predicted. 
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Final TER ratings also did not align with data from the SLO process. Analysis of pilot SLOs 
revealed that mid-fall teachers themselves classified 37% of students as underprepared to meet 
year-end goals. xxiii This suggests that it is unlikely that 91% of pilot teachers truly are Effective 
or Highly Effective and that TER ratings are inflated. However, the full picture of final teacher 
rubric ratings cannot be determined until after all evaluation measures are completed. 
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The final RISE teacher rubric ratings were also discouraging because of the lack of 
differentiation among competency ratings. The final competency ratings shown in Chart 3 
demonstrate that evaluators struggled to differentiate between teacher strengths and 
weaknesses. RISE evaluators rated almost all teachers as Effective or Highly Effective across all 
nine Domain 2 instructional competencies. xxiv In fact, a majority of all teachers (67%) were rated 
"effective" or higher on every competency on the RISE teacher rubric. In addition, evaluators 
rarely used the bottom two rating categories. No more than eleven percent (11%) of teachers 
were rated "improvement necessary" or "ineffective" on any skill. 

In theory, it is harder for teachers to master some instructional competencies than others. For 
example, developing students' higher-order thinking skills is generally considered a tough-to- 
master advanced teaching skill. Yet "Competency 2-6: Develop Higher Level of Understanding 
Through Rigorous Instruction and Work" had nearly one-fourth of teachers rated Highly 
Effective (23%) and over two-thirds rated Effective (68%). xxv 

Chart 3: A majority of teachers were rated Effective or Highly Effective on each 
instructional competency. 



Indiana's evaluation legislation, the RISE evaluation system, and most specifically the TER, 
were designed to help educators develop by helping administrators and teachers identify, and 
receive more feedback on, strengths and weaknesses. The TER was also designed to help 
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educators reverse the Widget Effect . A rubric with 
multiple performance levels allows evaluators to give, 
and teachers to receive, more specific and targeted 
evidence on instructional practice so that teachers can 
become better. Yet this only works if evaluators are 
able to identify and differentiate teacher strengths and 
weaknesses accurately. 


/ ^ 

Teachers who view their 

evaluator as an 
instructional leader are 
significantly more likely 
to be satisfied with the 
new evaluation system. 


Despite the significant shift in how evaluators are 
spending their time, the fact that evaluators struggle to 
identify teacher weaknesses compromises their own ability to become the instructional leaders 
of their schools. Overall, teachers report that they do not view their evaluators as instructional 
leaders and indicate that evaluators might need more time and training to develop into strong 
instructional leaders. Just over one-third of teachers agree that they view their evaluator as an 
instructional leader (37%). xxvi This has large implications for the success of a culture shift at the 
school-level, as teachers who do view their evaluator as an instructional leader are significantly 
more likely to be satisfied with the new evaluation system. xxvii 


Moving forward, evaluators will need to provide better, more actionable feedback. While 
nearly seven in ten teachers agreed that feedback was based on examples from classroom 
observations (69%), few teachers report that feedback was actionable or tied to development 
opportunities. Only four in ten teachers agreed feedback provided specific suggestions for what 
to change in their daily instruction (41%), and just over one third were pointed toward 
professional development opportunities based on areas of improvement (34%) or discussed a 
development plan for next year (37%). xxviii As one pilot teacher commented, "Don't even bother 
giving me feedback unless you can provide concrete strategies to help me improve." Teachers 
who received individualized professional development, or immediately actionable feedback, 
were significantly more likely to be satisfied with the new evaluation system. xxix 


Evaluators report that it is very hard to give teachers lower ratings than they have been given in 
the past. xxx They also noted that more training is needed or preferred on how to help teachers 
who performed poorly improve (73%), and on helping good teachers become great teachers 
(71%). xxx i 


Given evaluators' response that providing feedback on low ratings is difficult, this might 
explain part of the reluctance of evaluators to give low ratings and the inflation observed in the 
pilot. In addition, evaluators might have given higher-than-deserved ratings when they did not 
know what next steps or development to offer teachers to improve their practice. Collectively, 
this data indicates that teachers are not yet getting the differentiated feedback on their practice 
that is needed to improve instructional practice, school culture, and student achievement. 
Changes in how evaluators conduct observations and map observation evidence to the rubric. 
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increased central office support for evaluators, and more training is needed to develop 
administrators' ability to provide teachers with more accurate, actionable feedback. 

3. Collaboration and conversation promote satisfaction and student achievement. 

Survey and focus group findings show that the rubric and other elements of the new evaluation 
system can be used to promote student achievement, but those who collaborate and discuss 
these elements frequently are significantly more likely to make the connections to excellent 
teaching and student achievement, and to be satisfied with the system. When teachers, 
evaluators, and corporation leaders know the system and work together, they are more likely to 
be satisfied and students are more likely to achieve rigorous academic goals. 

One thing is clear across all six pilots: the new rubric helps promote a common language for 
instructional excellence. Collaborating around the new rubric at least 2-3 times per month 

increases teacher satisfaction with the new 
evaluation system and promotes a culture of 
student learning where expectations are clear and 
where teachers understand specifically how they 
can improve their instruction. Collaboration and 
conversation, particularly around how to use the 
instructional rubric to improve instruction, 
significantly increased agreement that the new 
evaluation system improves instruction and is good 
for student learning (21 percentage points more 
agreement for each). xxxii Additionally, teachers who 
were provided opportunities to collaborate on 
planning instruction or designing lessons aligned to 
the new rubric were significantly more satisfied 
with the new evaluation system overall (18 
percentage points more satisfied and 17 percentage points more satisfied, respectively) and 
more likely to think that their ratings were fair and accurate (12 percentage points more and 11 
percentage points more, respectively). xxxiii Additionally, multiple monthly opportunities to 
discuss how the instructional rubric can improve instruction significantly increased the 
likelihood that teachers discussed a development plan with their evaluator (24 percentage 
points more likely) . xxxiv 

Collaboration around measures of student learning also increased teacher satisfaction with the 
new evaluation system, and made better use of data in instruction. In RISE pilots, considerable 
time was spent on implementing SLOs. Teachers who had sufficient time to work together on 
aspects of the SLO process were significantly more likely to agree that the evaluation system 
encouraged data-driven instruction in their school (25 percentage points more likely). xxxv 
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When teachers, 

evaluators, and 
corporation leaders 
know the system and 
work together, they are 
more likely to be 
satisfied, and students 
are more likely to 
achieve rigorous 
academic goals. 
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Conversation around SLOs not only significantly increased satisfaction with the evaluation 
system and agreement that the system is good for student learning (13 and 18 percentage points 
more likely respectively), but also significantly increased the likelihood that teachers agreed 
time spent implementing the evaluation system was well spent (20 percentage points more 
likely) , xxxvi 


One area where pilot corporations all cited a need for 
improvement was principal-to-principal 

collaboration. Corporations and teachers both report 
a need for increased evaluator collaboration to build 
greater consistency within and across schools - 
consistency in knowledge about the system, 
consistency in number and length of observations, 
consistency in quality and timeliness of feedback, and 
consistency in using evidence to provide ratings. This was a challenge for both small and large 
pilot corporations, as teachers report inconsistencies across RISE evaluators even within the 
same building. A lack of consistency across evaluators could present a large barrier to a 
successful shift in culture and the belief among teachers that the new evaluation system is fair 
and accurate. Central office staff reflected that, moving forward, increased principal-to- 
principal collaboration and collaborative training is needed to improve evaluator consistency. 


One area where pilot 
corporations all cited a 
need for improvement 
was principal-to- 
principal collaboration. 


4. Measuring student growth and achievement needs to be prioritized by teachers and 
evaluators. 


Any good evaluation system should aim to directly measure a teacher's contribution to what 
matters most - student learning. As a result, new evaluation systems, and a successful shift in 
school culture, necessitate that corporations develop a new focus on student learning. The 
largest challenge to measuring student learning is presented by non-tested subjects and grades 
(NTSGs) — subjects and grades that are not measured by ISTEP growth model data. The RISE 
evaluation system uses SLOs as one measure of student learning in these subjects and grades, as 
well as in tested grades and subjects. The SLO process is built on the actions that great teachers 
take already: using data to consider student starting points, setting rigorous academic goals, 
and measuring and tracking progress toward those goals in order to adjust instructional 
practice. 

Ultimately, implementing SLOs was challenging but largely successful during the pilot year. 
Teachers also note that working on SLOs was the most time-consuming part of the new 
evaluation system, requiring four to six and a half hours. xxxvii Both RISE teachers and evaluators 
report that creating or updating assessments for rigor and alignment to standards required the 
most time. Over one third of teachers using teacher-created assessments spent five hours or 
more just on preparing these assessments. xxxviii Evaluators noted that approving teacher or 
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school developed assessments for SLOs took a median of 30 minutes per subject or grade, a 
considerable time commitment for all evaluators. xxxix 

Teachers overall thought every step of the process was challenging, but most commonly, 
teachers thought obtaining prior year student data in order to consider student starting points 
was very challenging (most commonly rated 7 on a 1 (not challenging) to 7 (very challenging) 
scale). xl Evaluators thought that overall, providing feedback on rigor of SLOs was most 

challenging (most commonly rated 5 on a 1 (not 
challenging) to 7 (very challenging) scale). xli 

Even though time-consuming and challenging, 
RISE pilot corporations did implement the SLO 
process as it was designed. As directed in the SLO 
Handbook and trainings throughout the year, 
almost all teachers used the most common 
assessment for their primary SLO when available 
(98% of teachers sampled), meaning that student 
learning was measured using the most appropriate 
assessment available. No analysis was done on the 
rigor of teacher-created or school-created 
assessments. However, most teachers and 
evaluators set content mastery at the state- 
determined level or at least 75 percent (94% of teachers sampled). Although given the flexibility 
to use an alternative measure of student learning for their secondary SLO if more applicable to 
the needs of low prepared students, most teachers used the same assessment (70% of teachers 
sampled). Almost two-thirds of teachers who provided information on how they leveled 
students used more than two data points to determine student starting points (62%). Also, nine 
out of ten teachers correctly targeted their low group of students when setting their secondary 
SLO (92%). xlii This is encouraging because it shows that the SLO process is ensuring that the 
needs of under-prepared students in a classroom are addressed. 

Overall, a majority of teachers (59%) and evaluators (65%) believe that SLOs should accompany 
other measures of student learning in an evaluation system in order to accurately reflect 
teachers' progress with their students. xliii RISE teachers who had sufficient time to work 
together on aspects of the SLO process were significantly more likely to think SLOs are a good 
measure of their students' learning (22 percentage points more likely), and as noted earlier, 
agree that the SLO process encourages data-driven instruction in their schools (25 percentage 
points more likely) . xliv 
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End-of-Year Recommendations 


Based on the findings above, consider these recommendations when implementing a new 
evaluation system. 

1. Prepare for a culture shift by aligning the priorities of central office staff, administrators, 
and teachers. 

Creating a culture shift across an entire corporation requires a shared vision of teaching 
excellence and an alignment of roles between central office staff, administrators, and teachers. 
A focus on student learning must drive all actions, especially in implementing the new 
evaluation system. To achieve success, corporations must own and support this shift locally. 

Ensure superintendent and central office buy-in. As a leader, the superintendent must be a 
supportive proponent of the new evaluation system. These specific steps were taken by 
superintendents across the pilots: 

• Visit each school and conduct observations in classrooms - One superintendent of a smaller 
pilot district conducted observations in every classroom this year. Even in larger pilot 
corporations, superintendents regularly visited classrooms. 

• Align the evaluation of principals, principals' supervisors , and the superintendent with that of 
teachers - Administrators, like teachers, need feedback and development. They also 
need to be held accountable for the quality of their instructional leadership and student 
achievement. Create consistency across the corporation by aligning systems for 
evaluating the superintendent, principals' supervisors, and principals with the teacher 
evaluation system. Consider the RISE Principal Evaluation tool or another tool that 
includes observable indicators of success as well as measures of student performance. 

• Participate in or lead evaluation design and implementation committees - Superintendents in 
small and large corporations will need to delegate some evaluation implementation 
responsibilities. However, it is important that superintendents and principals' 
supervisors are engaged in key design work, including listening to stakeholders and 
promoting a high-level view of how the new evaluation system aligns with other 
corporation priorities. 

• Meet regularly with administrators and teachers to discuss academic goals and progress - 
Superintendents and central office staff in all pilots met regularly with administrators to 
check-in on the progress and quality of observations and learning goals. One 
superintendent made it a priority to meet with as many teachers as possible early in the 
year to discuss academic goals. 
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Align evaluation work with the mission, goals, and success metrics of the district. In order 
for the evaluation system to support a culture shift, principals and teachers must see evaluation 
work as an integrated part of the actions they already take, or must take, to promote student 
learning. Corporations should embed evaluation work within the corporation's mission, goals, 
and success metrics - with the desired outcome of all corporation initiatives being increased 
student achievement. Evaluation-specific metrics might include quality of feedback, number of 
observations, final ratings distributions, and student performance on SLOs and ISTEP. 

Communicate to all stakeholders. Build a shared vision among parents, board members, 
students, and the community. Stakeholders need to understand why a specific evaluation 
system has been chosen, how the new evaluation system will align with other corporation 
priorities to promote student learning, and how the new evaluation system will change the way 
administrators spend their time. Pilot corporations report that a successful communication 
strategy will vary depending on the corporation size and local context. 

2. Create clear professional expectations and support new responsibilities. 

New evaluation systems can promote a shared vision of teaching excellence and increase 
teacher and administrator ownership over student learning. As teachers expect more frequent 
observation and feedback, administrators will need to learn new skills and change how they 
spend their time. Corporations will need to support this shift to build consistency across 
schools. 

Utilize the rubric to create a shared vision of excellence. Pilot corporation leaders and 
teachers mentioned repeatedly that the rubric should be used to create a shared vision of 
teaching excellence before all other evaluation priorities. This requires central office staff, 
administrators, and teachers to have a common understanding of rubric language. For this to 
succeed, the corporation will need to define locally exactly what each indicator looks like and 
sounds like in practice. 

• Create a shared understanding of each rubric indicator - Pilot corporations cited this as one 
area where training cannot be emphasized enough throughout the school year. Without 
extensive video libraries and prior training, building and communicating this common 
vision can be difficult. Here are some of the strategies that were used in the pilot 
corporations: 

o Designating a central office employee or outside consultant to help evaluators 
and teachers create a consistent understanding of the rubric by designing and 
facilitating training activities and communicating any corporation-specific 
expectations for individual rubric indicators. 


16 


o Encouraging central office-evaluator and evaluator-evaluator co-observations 
and observation debriefs. 

o Setting aside " sacred" time in administrator meetings to collaboratively observe 
and discuss instruction, as well as build evaluator consistency in mapping 
evidence to the rubric. 

o Having teachers practice observing and rating their peers in the classroom, 
o Utilizing online videos and observation tools, such as Observ Ability, to build 
evaluator skill. 

• When using RISE , set clear definitions for Core Professionalism at the corporation level - All 
parts of the new rubric should be commonly defined across the corporation, especially 
those areas containing only two performance categories. Because each Core 
Professionalism competency is high-stakes, RISE evaluators were hesitant to penalize 
teachers without a clear definition for what constitutes "Meets" or "Does Not Meet". xlv 
The most commonly cited area for corporations to define more specifically at the local 
level was Attendance. 

Provide all teachers with consistent observation and feedback. Teachers in the pilot report 
expectations for more frequent observations and increased feedback. Furthermore, 
implementation of the new evaluation system across pilots proved that the number of 
observations expected is possible. Careful planning is needed at the corporation level, however, 
to meet teacher expectations and to build consistency in the way teachers are being observed 
and feedback delivered. 

• Evaluators should utilize a mix of shorter observations (10-20 minutes) to supplement longer 
observations - Teachers find shorter observations acceptable as long as they are numerous 
or accompanied by longer observations. Teachers from five of the six pilot corporations 
report a preference for either five 20-minute observations per year, or three 10-minute 
observations and two 40-minute observations per year. xlvi Evaluators also favor three 10- 
minute observations and two 40-minute observations per year. xlvii Shorter observations 
can increase evaluators' capacity while still delivering accurate ratings. The Measures of 
Effective Teaching (MET) Project found that a 15-minute observation can be just as 
meaningful as an entire class period. xlviii 


• Provide evaluators flexibility, but ensure all teachers receive adequate, consistent observations - 
Extra observations can provide struggling teachers with more support. Evaluators 
should be given flexibility to vary the overall number of observations each teacher 
receives. All teachers, however, should receive consistent observations, regardless of 
performance level. Corporations also need to ensure that even those teachers who plan 
to retire receive observations and feedback. As professionals, all teachers deserve 
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regular feedback and evaluators should not be allowed to opt-out of observations for 
excellent or retiring teachers. 

Assist administrators' shift from building manager to instructional leader. A select group of 
administrators report spending 100% more time on evaluation and teacher development than in 
previous years. xlix Although pilot evaluators report that this shift seemed to be the correct 
reallocation of time, this transition took time and will require support from the central office. 

• Administrators' supervisors need to spend more time in schools - Administrators' supervisors 
should be in schools helping develop evaluator skills in observation and feedback to 
increase efficiency. They also need to help identify areas where evaluators can delegate 
or otherwise shift more time toward evaluation. By spending more time in schools, 
supervisors will be well informed to help build best practice and consistent procedures 
across all schools in the corporation. Central office staff in some pilot corporations 
maintained full-time offices in schools to support this shift. 

• Administrators' supervisors need to hold them accountable for accuracy of ratings and quality of 
feedback - Part of being an instructional leader is being able to deliver accurate, honest 
feedback about teachers' strengths and weaknesses. Teachers cannot develop if 
administrators are giving inflated ratings and feedback, which create inconsistency 
between schools and undermine any positive culture shift. If administrators struggle 
with the skills necessary to accurately rate teachers and provide actionable or difficult 
feedback, supervisors should intervene with co-observations and co-conferences. 

• Set clear expectations for administrators - Supervisors in one pilot corporation commented 
that administrators are expected to spend at least 75% of their time on classroom or 
academic activities. 1 Providing administrators with a clear expectation for time 
allocation can help build consistency within and across schools, as well as help 
administrators determine what activities to prioritize. 

• Corporations need to eliminate unnecessary administrator duties - One pilot superintendent 
commented that it doesn't make sense to pay someone an administrator's salary to have 
them perform lunch or bus duty. Corporations should alleviate administrators of 
obligations that don't directly support instruction by delegating administrative duties 
and paperwork to other staff or hiring additional staff to specifically perform these 
duties. Corporations can also help administrators build support in their school 
community by communicating to all stakeholders, especially parents, why 
administrators are more likely to be in classrooms observing and not always readily 
available for unannounced meetings. 
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3. Develop evaluator skills in identifying teacher strengths and weaknesses. 

Teachers consistently report that increased feedback on their instructional practice is one of the 
most important benefits of a new evaluation system. The pilot, however, found that evaluators 
struggled to differentiate between teacher strengths and weaknesses, as well as to provide 
teachers with specific steps to improve. To accurately and fairly differentiate teachers, as well 
as provide them the feedback they need to develop, corporations will need to develop evaluator 
skills. 

Corporations should provide collaborative development opportunities in observation and 
rating to evaluators. Pilot corporations report that rating teacher practice in a group through 
videos or co-observations was one of the most valuable training exercises for helping evaluators 
arrive at accurate ratings. However, as the pilot TER ratings distributions demonstrated, these 
basic trainings are not sufficient to accurately set expectations for what constitutes excellence . 11 
Much more extensive ongoing work and practice are required. Corporations can utilize an 
online training resource such as Observ Ability to help evaluators build consistency in taking 
classroom evidence and mapping evidence to a rubric but must institute ongoing opportunities 
for practice to continuously develop evaluator skills. 

Evaluators should map classroom observation evidence to rubric indicators immediately. 

Evaluators who map classroom evidence to the rubric quickly and frequently are able to more 
accurately rate teachers because the observation is fresh in their mind. In addition, frequent 
mapping of evidence allows the evaluator to keep a constant pulse on each individual teacher 
and staff as a whole. 

Corporations should collect teacher ratings on a frequent, ongoing basis. Corporations 
should require administrators to compile observation evidence and determine rubric ratings at 
least once before the end of the year, preferably at the midpoint of the school year. Compiling 
observation notes can help evaluators determine areas where more evidence needs to be 
collected, identify staff strengths and weaknesses for immediate development, and determine 
more accurate year-end ratings because there is less data to sift through. In addition, 
corporation leadership and evaluators should utilize mid-year ratings to analyze whether mid- 
year teacher ratings are aligned with current student progress. If they are not, corporations 
should intervene to provide evaluators more support in accurate observation and rating. 

Teachers should receive observations from more than one evaluator whenever possible. The 

MET Project found that multiple observers can increase the reliability of classroom observation 
ratings . 111 Multiple evaluators can provide more accurate ratings if evaluators are given time to 
discuss teacher observations and evidence with one another. Rather than assigning each 
teacher only one evaluator, divide up observations between evaluators so that the same teacher 
is observed by different people at different times. The logistics of this process, including 


19 


dividing observations and finding time for evaluators to discuss evidence collaboratively, 
proved difficult for many pilot schools. Corporations should determine a clear process for this 
work and explore technology solutions that allow evaluators to coordinate their schedules and 
to access observation evidence from other evaluators in the absence of a face-to-face meeting. 

Corporations should build evaluator skills in knowing what actionable feedback looks like 
and sounds like. Corporations can adopt a specific feedback framework, such as Bambrick's 
Framework mi , to provide its administrators with a clear, consistent model for providing 
feedback to teachers. Corporations should focus evaluator development on providing feedback 
to teachers through role play, co-observing and conferencing between the evaluator and his or 
her manager, videotaping evaluator-teacher conferences to promote evaluator reflection and 
analysis of practice, or teaching a specific feedback framework. All frameworks should explain 
that strong feedback is: 

• Timely and frequent 

• Based on specific observation evidence 

• Rooted in rubric-specific language 

• Focused on both strengths and weaknesses 

• Honest and transparent 

• Actionable, providing specific steps for improvement 

Corporations should utilize tools and strategies that help evaluators provide actionable 
development steps to teachers. Several pilot corporations found online professional 
development tools linked to the new rubric to be useful. In the absence of evaluator capacity to 
point teachers toward specific action steps for improvement, online video libraries and 
exemplar resources can help teachers to link feedback to development. Tools such as Doug 
Lemov's Teach Like A Champion Uv can help teachers determine practices that can immediately 
improve their instruction. When available, instructional coaches can provide teachers with 
additional help linked to their rubric strengths and weaknesses. 

4. Provide opportunities for collaboration around the new evaluation system. 

Increased collaboration can help build a strong, student-centered culture and promote the 
success of the new evaluation system. Collaboration also builds consistency of practice within 
and across schools. Therefore, numerous opportunities to collaborate should be provided by 
corporations and administrators. 

Make specific time for collaboration. The time and support required for successful 
implementation of a new evaluation system can be extensive and hard to schedule. During a 
school year that was planned prior to arrival of the new system, pilot corporations struggled to 
find time for teachers and evaluators to collaborate on the system. Having had the past year to 
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prepare for a new system, corporations across the state should have planned carefully to avoid 
facing the same challenge. Even at this juncture there are a number of things that can be done: 

• Build in extra time for collaboration at the beginning or end of the day - One pilot corporation 
built in collaborative development time for teachers, administrators, and instructional 
coaches at the start of every school day. This time could be allocated for collaboration 
on evaluation activities or toward targeted weaknesses identified in observations. 

• Integrate evaluation collaboration into already established processes - Shift topics or 
reprioritize evaluation in already scheduled meetings. Pilot corporations did this by 
determining how certain evaluation responsibilities could be aligned with the goals and 
activities of other corporation priorities. Almost all teachers have weekly or daily 
planning time in their schedules. Schedule planning time so that teachers in similar 
grades and subjects can collaborate on assessments or planning. One pilot corporation 
also continued to utilize the Professional Learning Community (PLC) process it had 
previously instituted to provide teachers the time they needed to plan common 
assessments. 

Promote collaboration through teacher development. Administrators should provide teachers 
with collaborative development opportunities based on their identified strengths and 
weaknesses. Collaborative development helps build a shared sense of teacher excellence and 
student achievement. 

• Use observation data to identify common trends in strengths and weaknesses - Pilot 
corporations successfully used observation analysis software to identify and develop 
needs for individual teachers, groups of teachers, and across entire schools. If software 
is not available, it becomes more important for evaluators to compile observation 
evidence and ratings on a regular basis as previously mentioned. As a result, 
immediate, collaborative intervention can be provided to individual teachers or entire 
staffs. But remember — if observations consistently rate most teachers as effective or 
highly effective, the system can't provide the necessary data to properly align 
development opportunities. 

• Pair teachers to increase the quality and efficiency of development - Having teachers with 
similar weaknesses work on development activities together can increase the strength 
and efficiency of development. Consider utilizing instructional coaches or group 
activities to target similar weaknesses. In addition, strong teachers can also be utilized 
to help build the skills of teachers in particular areas of weakness during common 
planning or school-wide meetings, giving them more of a leadership role and also 
building a shared sense of ownership among teachers. 
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• Provide teachers time to plan lessons and instruction together - As mentioned earlier, teachers 
who plan together are more likely to be satisfied with the new evaluation system and 
more likely to think the evaluation system is fair and accurate. Corporations should 
structure common planning times for teachers in similar grades and subjects. 

Use collaboration to increase expectations for student learning and data-driven decision- 
making. Teachers who collaborated on SLOs were more likely to report that time spent on the 
new evaluation system was well spent. In addition, collaboration on creating common 
assessments and setting student achievement targets creates a shared sense of accountability 
and increased reflection on data from which to make decisions. 

• Increase efficiency of the assessment creation and approval process - Provide time for teachers 
to work together to create common assessments and complete approval forms. If several 
teachers are teaching the same subject, they should be using the same assessment and 
setting the same content mastery goal for students. This will facilitate collaboration that 
will help build consistency of rigorous academic expectations across classrooms. 

• Promote the consistent use of data to determine student preparedness - Additionally, teachers 
of the same grade or subject may find it valuable to work together to identify data points 
that they plan to use to group students into levels of preparedness. While different 
teachers will have students at different levels of preparedness, they may all look at the 
same data points to group students. Building a centralized database of past student 
performance on standardized and teacher-created assessments will help to facilitate this 
activity as well as increase efficiency of the SLO process. 

• Cultivate shared data-driven planning and instruction - Corporations will need to support 
evaluator and teacher skills in using data to make data-driven decisions. To do this, 
pilot corporations either designated a central office staff member or outside consultants 
to provide training. Opportunities to collaboratively analyze assessment data, and to 
use that data to drive instructional planning and execution, creates a shared vision and 
ownership of student progress. Pilot corporations found it successful to have groups of 
teachers and administrators within and across schools collaborate together. 

5. Encourage rigorous measures of student learning. 

The student learning objective process can be both challenging and time consuming, but 
teachers and evaluators agree that when done properly it can be a valuable way to steer 
instruction and measure student learning. The recommendations in this section focus on tips 
for making the SLO process more efficient, as well as tips for writing rigorous SLOs. 
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Train teachers on the student learning objective process. Adequately train teachers on this 
process in the beginning of the school year (given the timing of the SLO process, this training 
should occur in the first week or so of school). At ESC RISE trainings, evaluators received 
" Meetings in a Box" to help train teachers on this process (contact your local ESC for more 
information) lv . Ensuring that teachers understand the process upfront will head-off much 
confusion, promote positive culture change, and save time throughout the process. 

Identify/Create assessments prior to the beginning of the school year, or immediately 
thereafter. Corporations were advised to evaluate their library of assessments this past spring 
through RISE training, DOE guidance, and the mid-year pilot report. The beginning of the 
school year can be a stressful time and spending this time creating end-of-course assessments 
(where none exist) is often not a priority. Assessments must be approved and in place in 
September before the rest of the SLO process can proceed. Use the assessment guidance on the 
DOE website to help guide this process. Corporations should also consider utilizing the soon- 
to-be released Indiana Course- Aligned Assessments to increase the coverage of assessments in 
non-tested grades and subjects. Assessment blueprints for these assessments are available 
already. Spending the time to identify and create good assessments this year will also save time 
in years to come. Assessments only have to be created once - if a good assessment of student 
learning is created, teachers can continue to use it for SLOs year after year. 

Identify technology solutions for storing student learning data in your corporation. The 

second step of the SLO process involves collecting prior year data on students in order to 
identify their level of preparedness for the course. Teachers in the pilot corporations found this 
part of the process to be particularly challenging when they did not have direct access to prior 
year student performance data. Student testing data, grades, GPAs, attendance data, etc. 
should be stored in a place that is secure, but easily accessible by teachers looking to find 
background information on their students. This can save time and frustration and lead to the 
use of more data points in determining student levels of preparedness. 

Thoroughly complete Steps 1 and 2 of the process before writing student learning objectives. 

The actual writing of the SLO is a very quick part of the process if the first two steps 
(assessment approval and determining student starting points) have been completed 
thoroughly following the RISE guidance. Step 3 (writing the objectives) should take no longer 
than a few minutes for teachers who have an already approved assessment with an approved 
content mastery score, as well as a completed grouping of students by levels of preparedness. 
Teachers who had not correctly completed Steps 1 and 2 of the process struggled with Step 3 
and spent an unnecessary amount of time here. 

Use conferencing time between teachers and evaluators wisely. There are many approval 
points within the SLO process. The assessment must be approved, the levels of student 
preparedness must be approved, and the SLOs must be approved. Evaluators and teachers do 
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not have to meet every step along the way. Take advantage of evaluation technology, email, or 
even snail mail to assist in the process. Face-to-face conversations might only be necessary 
when there seems to be a real disconnect between the evaluator and teacher's expectations. 
Some evaluators found it helpful to meet with teachers during department meetings for some 
approval conversations - especially where common assessments or data points were used for 
Step 1 and Step 2. 

Of course, this is not to say that evaluators and teachers should never meet regarding the SLO 
process. If teachers are struggling to meet expectations, this certainly may require an in-person 
conversation. Similarly, it is expected that, throughout the year, progress on SLOs will be a 
topic of conversation during post-observation conferences as well as staff-wide during whole- 
school meetings or professional development. 

Conclusion 

r 

No matter which system is adopted or what training takes 
place, successfully implementing a new teacher evaluation 
system will require a profound shift in school culture. For 
most educators, the new process represents a significant 
change in the way they spend their time and energy. In 
most schools, it will also require a deliberate shift in the 
relationship between teachers and administrators who will 
need to collaborate throughout the school year to identify 
strengths and weaknesses in the classroom. 

The change won't happen right away, but a year of pilot 
experiences suggests that with practice and commitment, 

Indiana educators can dramatically shift their school 
cultures and raise student achievement by prioritizing classroom visits and honest 
conversations about instruction and student learning. Implementing a new evaluation system 
is a corporation-wide effort that requires central office staff, administrators and teachers to align 
their priorities and commit to constant reflection and improvement. All staff will need to learn 
new skills and re-evaluate how they use their time and energy, but the end result is 
meaningfully improved instruction that will help Indiana schools fulfill their most important 
mission: preparing students for successful futures. 


Implementing a new 
evaluation system is a 
corporation-wide effort 
that requires central 
office staff, 
administrators and 
teachers to align their 
priorities and commit 
to ongoing reflection 
and improvement. 
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Appendix A-l 

Pilot Corporation Profiles 


Corp 

ID 

Corp Name 

Total 

Enrollment 

(11-12) 

American 

Indian 

Black 

Asian 

Hispanic 

White 

Multiracial 

FRL 

Pass 

ELA & 

Math 

ISTEP+ 

(10-11) 

Pass ELA 

& Math 
ECA (lO- 

ll) 

Grad 

Rate 

(2011) 

0235 

Fort Wayne Community 
Schools 

30821 

0.6% 

24.7% 

4.2% 

13.8% 

49.1% 

7.6% 

69.2% 

63.0% 

49.7% 

88.1 

2920 

Bloomfield School District 

1035 

0.3% 

0.7% 

0.6% 

2.4% 

93.7% 

2.3% 

37.5% 

69.4% 

70.9% 

97.2 

1730 

Greensburg Community 
Schools 

2294 

0.3% 

0.0% 

0.8% 

1.7% 

95.2% 

2.0% 

50.7% 

79.4% 

65.9% 

96.4 

5360 

MSD Warren Township 

11899 

0.2% 

46.2% 

1.0% 

10.0% 

35.6% 

7.0% 

66.5% 

62.4% 

54.7% 

87.6 

5380 

Beech Grove City Schools 

2734 

1.0% 

7.3% 

0.7% 

5.5% 

79.6% 

6.0% 

63.6% 

72.5% 

67.5% 

84.0 

5480 

Bremen Public Schools 

1461 

0.0% 

0.3% 

0.6% 

21.4% 

76.1% 

1.5% 

39.1% 

79.2% 

75.2% 

87.8 


Notes: 


• The passing percentages provided for ECA are for first-time test takers. 

• Enrollment data are from SY 11-12, while ISTEP+, ECA and grad rates are from 2010-11; 
these are the most recent data available for each data element. 


Appendix A-2: Descriptions of Pilot Evaluation Systems 

RISE (Bloomfield, Fort Wayne, Greensburg) 

RISE consists of two main components. Professional Practice and Student Learning Measures, 
which are combined to determine a teacher's summative rating. 

The Professional Practice component is measured by the Indiana Teacher Effectiveness Rubric, 
which consists of four domains: Planning, Instruction, Leadership, and Core Professionalism. 
Evidence of a teacher's Professional Practice is collected during a minimum of two formal forty - 
minute observations and three short twenty-minute observations, although some pilot 
corporations choose to do more. All observations must be followed by written feedback, and 
long observations are also followed by required post conferences. Observations are performed 
by primary and secondary evaluators. All pilot corporations are utilizing building 
administrators only as primary and secondary evaluators this year. 

Student Learning Measures are comprised of three pieces of data: Individual Teacher Growth 
Model based on ISTEP scores (Grades 4-8 ELA and Math), School Wide Learning based on the 
A-F Accountability Policy, and Student Learning Objectives, teacher-written goals around 
rigorous assessments. 

A teacher receives a 1-4 rating for Professional Practice and each of the three Student Learning 
Measures. Those ratings are rolled into an overall summative rating. Depending on a teacher's 
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mix of classes. Professional Practice comprises 50-75% of a teacher's summative rating, and 
Student Learning Measures comprise the remaining 25-50%. 

For more information about RISE, please visit www.riseindiana.org 

TAP System (Beech Grove) 

The evaluation component of the TAP System considers both classroom lesson evaluations and 
student achievement growth measures. The classroom lesson evaluations are measured using 
the TAP rubric which includes four domains: Designing and Planning Instruction, Instruction, 
The Learning Environment, and Responsibilities. Those four domains are further defined by 19 
areas of effective instructional practice and an annual survey leading to an annual overall 
"Skills, Knowledge, and Responsibilities" (SKR) score. The TAP System requires 4-6 formal 
evaluations each year, two announced and two unannounced. Announced formal evaluations 
include a pre-conference and all four formal evaluations have post-conference and self- 
evaluation components. Observations are performed by master teachers, mentor teachers, and 
one or more administrators throughout the year. 

Student achievement growth measures include both individual teacher/ classroom-level data 
(when available) and school-level data. When determining both differentiated levels of teacher 
effectiveness and also performance-based compensation amounts, 50% of a teacher's rating is 
based on their annual SKR score and 50% is based on student achievement growth measures. 

More information on the TAP evaluation system can be found at: www.tapsystem.org 

Bremen Evaluation System (Bremen) 

Bremen's evaluation system consists of two main components, an assessment of a teacher's 
Professional Skills and Measures of Student Learning, which combine to determine a teacher's 
summative rating. 

Professional Skills are measured by the Bremen/ McREL teacher evaluation rubric which 
consists of five standards: Leadership, Respect, Knowledge of Content Taught, Instruction, and 
Reflection on Practice. Evidence of a teacher's Professional Skills is collected during a 
minimum of three or four forty-minute observations. A pre-conference is required prior to the 
first formal observation, but not required for future formal observations. A post-conference is 
required following each formal observation. Observations are performed by building 
administrators. 

Measures of Student learning are comprised of four pieces of data: school-wide growth based 
on ISTEP and ECA assessments, school-wide achievement based on ISTEP and ECA 
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assessments, individual growth based on locally created assessments, and individual 
achievement based on locally created assessments. 

A teacher receives an overall rating for Professional Skills and an overall rating for Measures of 
Student Learning. Those ratings are rolled up into an overall summative rating. Professional 
Skills comprise 75% of the summative rating, and Measures of Student Learning comprise 25% 
of the summative rating. 

MSD Warren Evaluation System (MSD Warren) 

MSD Warren is in the process of finalizing the design of its teacher evaluation system. It 
contains two main components, a measure of a teacher's Professional Practice and Measures of 
Student Learning, which combine to determine a teacher's summative rating. 

The Professional Practice component is measured by a teacher effectiveness rubric still in 
development. Currently, the rubric consists of four domains: Instructional Planning, Effective 
Instruction, Classroom Environment, and Professional Commitment. Evidence of a teacher's 
Professional Practice is collected during a minimum of four long, class-length observations and 
eighteen short five to seven minute observations. One long observation is announced, with a 
pre and post conference. Three long observations are unannounced and followed by written 
feedback. Short observations are followed by written feedback. Observations are performed by 
building administrators. 

Measures of Student Learning are comprised of two pieces of data: Individual Growth Data and 
School-wide Growth Data. The means by which these two pieces will be measured is still in 
development. 

A teacher receives a rating for each of the three individual rubric domains and the two 
Measures of Student Learning. Those ratings are rolled into an overall summative rating. 
Professional Practice comprises 70% of a teacher's summative rating, and Measures of Student 
Learning comprise the remaining 30%. 


Comparison of Systems 



RISE (Bloomfield, Fort 
Wayne, Greensburg) 

TAP (Beech Grove) 

Bremen 

MSD Warren 

Minimum Observations 

2 40-minute observations 

3 10-minute observations 

*ln addition to the RISE 
minimum requirements, 

Fort Wayne Community 
Schools requires daily 1-3 
minute snapshots 

4 formal observations (2 
announced, 2 
unannounced) 

3-4 40-minute 

observations 

4 class-length 
observations (1 
announced, 3 
unannounced) 

1 5-7minute observation 
once every three weeks 
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Observation Rubric used 

Indiana Teacher 

Effectiveness Rubric 

• Planning (10% of 
rubric score) 

• Instruction (75% of 
rubric score) 

• Leadership (15% of 
score) 

• Core Professionalism 
(factored after other 
three domains are 
rolled up) 

*Overall rubric score 

calculated before being 

weighted into summative 

TAP Rubric 

Designing and Planning 
Instruction (15% of rubric 
score) 

Learning Environment (5% 
of rubric score) 

Instruction (75% of rubric 

score 

Responsibilities (5% of 
rubric score) 

*These weightings are 
different for TAP Master 
and Mentor teachers to 
reflect their specific job 
responsibilities. 

Bremen/McREL Rubric 

• Leadership (17.5% of 
rubric score) 

• Respect (17.5% of 
rubric score) 

• Knowledge of 

Content Taught 
(17.5% of rubric 
score) 

• Instruction (30% of 
rubric score) 

• Reflection on 

Practice (17.5% of 
rubric score) 

*Overall rubric score 

calculated before being 

weighted into summative 

District revised Teacher 
Effectiveness Rubric 

• Instructional 

Planning (10% of 
summative score) 

• Effective Instruction 
(50% of summative 
score) 

• Classroom 
Environment (10% 
of summative score) 

• Professional 

Commitment 
(factored after other 
three domains) 

**Overall rubric score not 
calculated before being 
weighted into summative 

Student Data used 

Individual Growth Model 

Individual Growth Model 

School-wide growth (15% 

School-wide growth (10% 


Data (Grades 4-8 ELA and 

Data (Grades 4-8 ELA and 

of data score) 

of summative score) 


Math) (20-35% of 

Math) (30% of summative 




summative score) 

evaluation where 

School-wide achievement 

Individual teacher student 



available) 

(15% of data score) 

growth (20% of 


School Wide Growth 



summative score) 


Measure - based on A-F 

School Wide Growth 

Individual teacher student 



accountability policy (5% 

Measure (20-50% of 

growth (30% of date 



of summative score) 

summative evaluation) 

score) 



Individual Student 


Individual teacher student 



Learning Objectives (10- 


achievement (30% of data 



20% of summative score) 


score) 



*Overall student learning 


**Overall student 

*Overall student learning 


not score calculated 


learning score calculated 

score not calculated 


before being weighted 


before being weighted 

before being weighted 


into summative. 


into summative. 

into summative. 

% of Summative 

50-75% depending on a 

50% 

75% 

70% 

Evaluation based on 

teacher's classes 




Rubric Score 





% of Summative 

25-50% depending on a 

50% 

25% 

30% 

Evaluation based on 

teacher's classes 




Student Data 






Endnotes 


' TNTP, a national nonprofit organization founded by teachers, works with schools, districts and states to advance 
policies and practices that ensure effective teaching in every classroom. TNTP partnered with the IDOE to provide 
direct support to the three pilot corporations implementing RISE this year. The findings in this report are based on 
TNTP's analysis of data collected from all six pilot corporations. Recommendations are based not only on Indiana 
pilot findings, but on TNTP's experience doing similar work with other states and districts nationwide. 
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" Although changes, based on pilot findings, will be made to RISE 1.0 resulting in the release of RISE 2.0 in early 
August 2012, no changes to the system will be made that will require evaluators to receive additional training or 
retraining. 

Corporations will need to have plans in place for dealing with challenges that might arise in providing all certified 
staff with full evaluations, especially in situations where full data might not be available for a summative rating 
determination as designed by the new evaluation system (i.e. missing SLO or GM scores). Guidance will be 
provided by the IDOE. 

IV A recent TNTP report. Greenhouse Schools: How Schools Con Build Cultures Where Teachers and Students Thrive 
(2012), found that teachers value schools with a strong instructional culture, where faculty share a clear vision of 
excellent instruction, where schools place an emphasis on student learning, and where school leaders focus on 
helping all teachers reach their full potential in the classroom, largely through increased observation and feedback. 
Schools with stronger instructional cultures retain more of their top teachers and show greater student proficiency 
rates in reading and math. 

v Corporations should utilize the recommendations in this report, as well as additional resources, to help 
evaluators plan for providing all teachers with the new system's required number of observations. Although pilot 
teachers, on average, reported having received the required number of observations, corporations should plan for 
how to deal with situations where teachers receive so few observations that a lack of evidence makes it difficult to 
provide an accurate rubric rating. 

Vl Teachers were asked "How many times total did your evaluator(s) observe you this year?" and all respondents 
except for one corporation responded with a mode of 4 observations, while the other corporation, which had an 
expectation of more frequent short observations, responded with a mode of 35 observations (n=1558). Source: 
Indiana Pilot Corporations End-of-Year Teacher Survey, administered 5/14/12-5/25/12. 

v " Teachers were asked "To the best of your recollection, how many of those observations were for less than 10 
minutes, were for 10-20 minutes, were for 20-30 minutes, were for 30-40 minutes, or were for 40 minutes or 
more?" and all respondents except for one corporation responded with a mode of 2 10-20 minute observations 
and 2 40 minutes or more observations, while the other corporation, which had an expectation of more frequent 
short observations, responded with a mode of 15 less than 10 minute observations, 2 10-20 minute observations, 
and 2 20-30 minute observations (n=1558). Source: Indiana Pilot Corporations End-of-Year Teacher Survey, 
administered 5/14/12-5/25/12. 

Vl " Teachers were asked, "To the best of your recollection, how many of those observations included a pre- 
conference with your evaluator before the observation, included a post-conference with your evaluator within one 
week of the observation, were followed by written feedback from your evaluator based on what he/she saw in 
your classroom, were announced, and were unannounced?" and all respondents responded with a mode of 2 
including post-conference(n=1585), 15 followed by written feedback (n=1609), and 15 unannounced (n=1600). 
Source: Indiana Pilot Corporations End-of-Year Teacher Survey, administered 5/14/12-5/25/12. 

IX Teachers were asked on both Beginning-of-Year and End-of-Year surveys "How many observations were followed 
by written feedback from your evaluator based on what he/she saw in your classroom." When compared, the 
mean of observations that included written feedback were 37% at the beginning of the year(n=964), and 70% at 
the end of the year (n=1520). Source: Indiana Pilot Corporations End-of-Year Teacher Survey, administered 
5/14/12-5/25/12; and Indiana Pilot Corporations Beginning-of-Year Teacher Survey, administered 8/22/11- 
8/29/11. 

x 69% of teachers responded agree (strongly agree, agree) when asked their level of agreement with the statement 
"The feedback I received was always based on examples (or evidence) or what my evaluator observed in my 
classroom (n=1535)." Source: Indiana Pilot Corporations End-of-Year Teacher Survey, administered 5/14/12- 
5/25/12. 

Xl 57% of teachers responded agree (strongly agree, agree) when asked their level of agreement with the 
statement "Compared to the last time I was evaluated, I now expect to take a more active role in my own 
evaluation and development (n=1460)." Source: Indiana Pilot Corporations End-of-Year Teacher Survey, 
administered 5/14/12-5/25/12. 
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x " 51% of teachers responded agree (strongly agree, agree) when asked their level of agreement with the 
statement "I feel more accountable for student achievement now than in the past (n=1461)." Source: Indiana Pilot 
Corporations End-of-Year Teacher Survey, administered 5/14/12-5/25/12. 

Xl " 45%, 51%, and 47% of teachers responded agree (strongly agree, agree) respectively when asked their level of 
agreement with the statements "I consider the new teacher evaluation system's observation rubric/framework 
when planning lessons on a day-to-day basis (n=1650)," "I often reflect on my own instruction and consider how it 
fits within the new teacher evaluation system's observation rubric/framework(n=1651)," and "I consider the new 
teacher evaluation system's observation rubric/framework when seeking ways to improve my practice(n=1648)." 
Source: Indiana Pilot Corporations Mid-Year Teacher Survey, administered 1/16/12-1/27/12. 

XIV 85% of evaluators responded net agree (strongly agree, agree, somewhat agree) when asked their level of 
agreement with the statement "I use the new system's observation rubric/framework to guide my classroom 
observations." (n=133.) Source: Indiana Pilot Corporations Mid-Year Evaluator Survey, administered 1/16/12- 
1/27/12. 

xv 96% and 93% of evaluators responded agree (strongly agree, agree) respectively when asked their level of 
agreement with the statements "I provided feedback on my teachers' professional strengths that was aligned with 
the observation rubric/framework (n=97)," and "I provided feedback about specific areas aligned with the 
observation rubric/framework that my teachers can improve on next year (n=97)." Source: Indiana Pilot 
Corporations End-of-Year Evaluator Survey, administered 5/14/12-5/25/12. 

XVI 58% and 85% of evaluators responded agree (strongly agree, agree) respectively when asked their level of 
agreement with the statements "As a result of the new evaluation system, my responsibilities have shifted from 
non-instructional activities (such as lunch duty, bus duty, or hall monitoring) toward spending more time in the 
classroom and conferencing with teachers (n=126)," and "I believe that time spent on evaluation work was well- 
spent (n=130)." Source: Indiana Pilot Corporations End-of-Year Evaluator Survey, administered 5/14/12-5/25/12. 
xv " 89% of evaluators responded agree (strongly agree, agree) when asked their level of agreement with the 
statement "I believe that I have spent more time this year observing instruction and giving feedback to teachers 
than in previous years (n=127)." Source: Indiana Pilot Corporations End-of-Year Evaluator Survey, administered 
5/14/12-5/25/12. 

XVI " Source: Focus Groups and individual conversations at Pilot Corporations. 

XIX Source: Individual conversations with 5 Evaluators at Pilot Corporations. 
xx Source: Focus Groups and individual conversations at Pilot Corporations. 

XXI In addition, 2011 A-F school ratings show that RISE corporation schools were not all rated A, some even received 
Ds; therefore it is unlikely that so few teachers are actually Improvement Necessary and Ineffective when rated 
across the 19 TER competencies and the Core Professionalism Domain. 

xx " Sources: RISE professional practice ratings, collected May 2012; and Indiana Pilot Corporations End-of-Year 
Evaluator Survey, administered 5/14/12-5/25/12. 

XXI " Source: RISE Student Learning Objectives sample, collected January-February 2012. All SLOs collected from two 
smaller corporations were analyzed, along with a representative sample from a larger corporation. In all, 1,551 
SLOs were collected, and 421 (27%) were analyzed. 

XXIV Source: RISE professional practice ratings, collected May 2012. 
xxv Source: RISE professional practice ratings, collected May 2012. 

XXVI 37% of teachers responded agree (strongly agree, agree) when asked their level of agreement with the 
statements "I now view my evaluator as more of an instructional leader than ever before (n=1477)." Source: 
Indiana Pilot Corporations End-of-Year Teacher Survey, administered 5/14/12-5/25/12. 

xxv " Teachers who responded agree (strongly agree, agree) when asked their level of agreement with the statement 
"I now view my evaluator as more of an instructional leader than ever before" were 37 percentage points more 
likely to have responded favorable (strongly favorable, somewhat favorable) when asked "Do you have a strongly 
favorable, somewhat favorable, somewhat unfavorable, or strongly unfavorable opinion of the new teacher 
evaluation system?" (n=644). Source: Indiana Pilot Corporations End-of-Year Teacher Survey, administered 
5/14/12-5/25/12. 

XXVI " 69%, 41%, 34%, and 37% of teachers responded agree (strongly agree, agree) respectively when asked their 
level of agreement with the statements "The feedback I received was always based on examples (or evidence) or 
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what my evaluator observed in my classroom (n=1535)," "My evaluator provided specific suggestions for what I 
could immediately change in my daily instruction based on my areas for improvement (n=1404)," "My evaluator 
pointed me towards professional development opportunities based on my areas for improvement (n=1369)," "My 
evaluator and I discussed a professional development plan for next year (n=1147)." Source: Indiana Pilot 
Corporations End-of-Year Teacher Survey, administered 5/14/12-5/25/12. 

XXIX Teachers who responded agree (strongly agree, agree) when asked their level of agreement with the 
statements "My evaluator and I discussed a professional development plan for next year" and "My evaluator 
provided specific suggestions for what I could immediately change in my daily instruction based on my areas for 
improvement" were 18 and 25 percentage points more likely respectively to have responded favorable (strongly 
favorable, somewhat favorable) when asked "Do you have a strongly favorable, somewhat favorable, somewhat 
unfavorable, or strongly unfavorable opinion of the new teacher evaluation system?" (n=523) and (n=603). Source: 
Indiana Pilot Corporations End-of-Year Teacher Survey, administered 5/14/12-5/25/12. 
xxx Evaluators were asked "On a scale of one to seven where one means not challenging and seven means very 
challenging, please rate the following types of conversations," and "Giving teachers ratings that were lower than in 
the past" had the highest mean response (4.33, n=112). Source: Indiana Pilot Corporations End-of-Year Evaluator 
Survey, administered 5/14/12-5/25/12. 

XXXI 73% and 71% of evaluators responded either 'I need more training' or 'More training would be helpful, but is 
not necessary' when given the following statements respectively: "Helping teachers I've observed performing 
poorly improve (n=134)" and "Helping good teachers I've observed become great teachers (n=134)". Source: 
Indiana Pilot Corporations Mid-Year Evaluator Survey, administered 1/16/12-1/27/12. 

xxx " Teachers who responded at least 2-3 times per month (Almost daily, 1-2 times per week, 2-3 times per month) 
when asked how often they "Had conversations with colleagues, administrators, or central office leaders about 
how to use the observation rubric/framework to improve my instruction" were 21 percentage points more likely 
than those with less frequent conversations to have responded agree (strongly agree, agree) to "Overall, the new 
teacher evaluation system is good for my students' learning. (n=370)" and "Overall, the new teacher evaluation 
system will help me improve my instruction. (n=311)." Source: Indiana Pilot Corporations End-of-Year Teacher 
Survey, administered 5/14/12-5/25/12. 

XXXI " Teachers who responded at least 2-3 times per month (Almost daily, 1-2 times per week, 2-3 times per month) 
when asked how often they "Had conversations with colleagues, administrators, or central office leaders about 
how to use the observation rubric/framework to improve my instruction" and "Used the rubric/framework to help 
design instruction or create lessons with a colleague, administrator, or central office leader" were 18 and 17 
percentage points more likely respectively to have responded favorable (strongly favorable, somewhat favorable) 
when asked "Do you have a strongly favorable, somewhat favorable, somewhat unfavorable, or strongly 
unfavorable opinion of the new teacher evaluation system?" (n=666) and (n=667), and were 12 and 11 percentage 
points more likely respectively to have responded agree (strongly agree, agree) to "I am confident that the new 
teacher evaluation system provides teachers with ratings that are fair (n=216)" and "I am confident that the new 
teacher evaluation system provides teachers with ratings that are accurate (n=208)." Source: Indiana Pilot 
Corporations End-of-Year Teacher Survey, administered 5/14/12-5/25/12. 

XXXIV Teachers who responded at least 2-3 times per month (Almost daily, 1-2 times per week, 2-3 times per month) 
when asked how often they "Had conversations with colleagues, administrators, or central office leaders about 
how to use the observation rubric/framework to improve my instruction" were 24 percentage points more likely to 
have responded agree (strongly agree, agree) when asked their level of agreement with the statement "My 
evaluator and I discussed a professional development plan for next year (n=397)." Source: Indiana Pilot 
Corporations End-of-Year Teacher Survey, administered 5/14/12-5/25/12. 

xxxv Teachers who responded agree (strongly agree, agree) when asked their level of agreement with the statement 
"My colleagues and I were provided sufficient time to work together on aspects of the student learning objectives 
process" were 25 percentage points more likely to have responded they agree "The process of setting student 
learning objectives was good for helping encourage data-driven instruction in my school (n=357)." Source: Indiana 
Pilot Corporations End-of-Year Teacher Survey, administered 5/14/12-5/25/12. 

XXXVI Teachers who responded agree (strongly agree, agree) when asked their level of agreement with the statement 
"I had conversations with other teachers, administrators, or central office leaders to help determine my students' 
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level of preparedness" were 13 percentage points more likely to have responded favorable (strongly favorable, 
somewhat favorable) when asked "Do you have a strongly favorable, somewhat favorable, somewhat unfavorable, 
or strongly unfavorable opinion of the new teacher evaluation system? (n=392)," 18 percentage points more likely 
to agree "Overall, the new teacher evaluation system will help me improve my instruction (n=174)," and 20 
percentage points more likely to agree "Time spent on evaluation work is well-spent (n=176)." Source: Indiana 
Pilot Corporations End-of-Year Teacher Survey, administered 5/14/12-5/25/12. 

xxxv " Non-RISE Teachers spent a median of 4 hours on measuring student learning; RISE Teachers spent a 
cumulative median of 6.5 hours on measuring student learning. Source: Indiana Pilot Corporations Mid-Year 
Teacher Survey, administered 1/16/12-1/27/12. 

XXXVI " Teachers who responded yes to the question "For either your primary or secondary Student Learning 
Objective, are you using an assessment that you or teachers in your corporation created" spent 5 or more hours 
"preparing a teacher-created assessment for SLOs" (n=806). Source: Indiana Pilot Corporations Mid-Year Teacher 
Survey, administered 1/16/12-1/27/12. 

XXXIX Evaluators were asked how long "Evaluating and providing feedback on an assessment for the purposes of 
Student Learning Objectives to make sure it was of sufficient quality" took and responded a median of 30 minutes 
per teacher (n=64). Source: Indiana Pilot Corporations Mid-Year Evaluator Survey, administered 1/16/12-1/27/12. 
xl When asked "On a scale of one to seven where one means not challenging and seven means very challenging, 
please rate the following parts of the student learning objectives process," teachers responded with a mode of 7 
for "Obtaining prior year data to group my students into levels of preparedness." (n=780). Source: Indiana Pilot 
Corporations End-of-Year Teacher Survey, administered 5/14/12-5/25/12. 

xli 

When asked "On a scale of one to seven where one means not challenging and seven means very challenging, 
please rate the following activities," evaluators responded with a mode of 5 for "Providing feedback on rigor of 
student learning objectives." (n=73). Source: Indiana Pilot Corporations End-of-Year Evaluator Survey, 
administered 5/14/12-5/25/12. 

xl " Source: RISE Student Learning Objectives sample, collected January-February 2012. All SLOs collected from two 
smaller corporations were analyzed, along with a representative sample from a larger corporation. In all, 1,551 
SLOs were collected, and 421 (27%) were analyzed. 

xh " 59% of teachers and 65% of evaluators responded agree (strongly agree, agree) respectively when asked their 
level of agreement with the statement "I believe that student learning objectives should accompany other 
measures of student learning in an evaluation system in order to accurately reflect my progress with my students 
(n=975) and (n=68)." Source: Indiana Pilot Corporations End-of-Year Teacher Survey, administered 5/14/12- 
5/25/12; and Indiana Pilot Corporations End-of-Year Evaluator Survey, administered 5/14/12-5/25/12. 
xhv Teachers who responded agree (strongly agree, agree) when asked their level of agreement with the statement 
"My colleagues and I were provided sufficient time to work together on aspects of the student learning objectives 
process" were 25 percentage points more likely to have responded they agree "The process of setting student 
learning objectives was good for helping encourage data-driven instruction in my school (n=357)" and 22 
percentage points more likely to have responded they agree "I felt that my student learning objectives were a 
good measure of my students' learning (in one class) (n=280)." Source: Indiana Pilot Corporations End-of-Year 
Teacher Survey, administered 5/14/12-5/25/12. 

xlv Source: Focus Groups and individual conversations at Pilot Corporations. 

xlvi Teachers were asked "If you had the choice between the following types of observations throughout the course 
of a year, which do you believe would lead to the most valuable feedback and represent the most accurate 
reflection of your practice over time?," and five out of six corporations indicated their top two preferences as five 
20 minute observations per year (27%, 30%, 32%, 25%, and 37%) or three 10 minute observations and two 40 
minute observations (40%, 37%, 26%, 48%, 29%); the one corporation that set an expectation for more frequent 
shorter observations had a preference of one walk-though lasting 3 minutes or less and one 10 minute observation 
(n=1617). Options provided included "weekly walk-throughs lasting 3 minutes or less, one walk-through lasting 3 
minutes or less and one 10 minute observation, five 20 minute observations per year, or three 10 minute 
observations and two 40 minute observations." Source: Indiana Pilot Corporations End-of-Year Teacher Survey, 
administered 5/14/12-5/25/12. 
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xlv " 47% of evaluators chose three 10 minute observations and two 40 minute observations when asked "If you had 
the choice between the following types of observations throughout the course of a year, which do you believe 
would lead to the most valuable feedback and represent the most accurate reflection of your practice over time?" 
(n=127). Options provided included "weekly walk-throughs lasting 3 minutes or less, one walk-through lasting 3 
minutes or less and one 10 minute observation, five 20 minute observations per year, or three 10 minute 
observations and two 40 minute observations." Source: Indiana Pilot Corporations End-of-Year Evaluator Survey, 
administered 5/14/12-5/25/12. 

xlvi " TNTP, January 2012. " 'MET Made Simple: Building Research-Based Teacher Evaluations ." 
xhx Source: Focus Groups and individual conversations at Pilot Corporations. 

1 Source: Focus Groups and individual conversations at Pilot Corporations. 
h Source: RISE professional practice ratings, collected May 2012. 

MET Project Bill & Melinda Gates Foundation. January 2012. " Gathering Feedback for Teaching ." 

To learn more about and access the Bambrick feedback framework, contact Paul Bambrick-Santoyo or 
Uncommon Schools (http://www.uncommonschools.org/bio/1017/paul-bambrick-santoyo). 
hv Lemov,Doug. 2010. Teach Like a Champion: 49 Techniques that Put Students on the Path to College. Jossey-Bass. 
Iv Visit the Indiana Department of Education website to find contact information for all Education Service Centers: 
http://www.doe.in.gov/idoe/education-service-centers . 


33 


